1. Field of the Invention
The present invention relates to an image encoding/decoding apparatus, and more particularly to lossy coding with respect to multi-level input images.
2. Description of the Related Art
Since images generally comprise very large volumes of data, the images are generally compressed by encoding during storage and transmission. If the image data subject to image encoding at that time is largely classified into two types, the image data can be classified into, for example, natural images and artificial images.
The former type is one in which actually existing images have been converted into digital data by some means. For instance, an image which is obtained by reading a photograph by a scanner or by capturing a scene by a digital camera corresponds to this type. The latter type is one in which images which do not actually exist are generated as digital data by some means. For instance, computer graphics and a document which is prepared by a word processor correspond to this type. Hereafter, natural images and artificial images are used under these definitions.
Generally, as for natural images, noise tends to be superposed thereon during digital transform, and their high-frequency components tend to degrade. As a result, the resultant data has a large amount of information in low-order bits, and the number of colors used is also large. In addition, if natural images are subjected to frequency analysis, components are liable to concentrate on a low-frequency region, and a high-frequency region attenuates.
On the other hand, in the case of artificial images, the amount of information in low-order bits is not large excluding a case in which noise is intentionally added thereto, and colors which are used are also liable to concentrate on particular colors. In addition, since edges, fine lines, and the like are depicted sharply, a large amount of important information is included in a high-frequency region as well.
Two experimental examples for confirming the above fact are shown in FIGS. 26 to 18. As a first experiment, values in which the square roots of mean squares of coefficients obtained by discrete cosine transform (DCT) processing were individually determined were examined with respect to a number of images. The results in which the square roots were added for the respective eight areas shown in FIG. 26 are shown in the part b) of the same drawing. Since the DCT coefficients are expressed in such a manner that the frequency increases from upper left toward lower right, in FIG. 26 the right-hand side of the x-axis corresponds to a high frequency. As is apparent from the drawing, in the case of natural images, components decrease as the frequency becomes higher, whereas, in the case of artificial images, components are distributed in spite of the frequency.
In a second experiment, adjacent pixel values were fetched from an image, and the statistics of the result of subtraction of a left-hand pixel value from a right-hand pixel value was gathered. Hereinafter, this value is referred to as xe2x80x9ca previous value differentialxe2x80x9d. FIG. 28 shows the results of the second experiment. As is apparent from the drawing, in artificial images, the previous value differential is concentrated in 0 in comparison with natural images. This shows that the prediction accuracy in the prediction of the previous value for predicting the right-hand pixel value from the left-hand pixel value becomes high.
Hereafter, image encoding techniques which are effective for natural images and artificial images will be respectively described as first and second conventional examples.
First, a description will be given of a conventional encoding technique with respect to natural images as a first conventional example. Since a natural image inherently contains a large amount of information, it becomes necessary to quantize the information by some technique. Therefore, if consideration is given to the efficiency of quantization, since, in the case of a natural image, frequency components are concentrated in a low-frequency region, quantization in which average errors are made small can be realized by quantizing a low-frequency region finely and quantizing a high-frequency region coarsely. That is, it is possible to minimize the effect on image quality and reduce the amount of information efficiently.
Frequency transform coding, which is one technique of image encoding, makes use of this characteristic, effects frequency transform of an input image, and coarsely quantizes information in high-frequency, in particular. As a typical example of frequency transform coding, it is possible to cite the DCT method of Joint Photographic Experts Group (JPEG), which is an international standard. Hereafter, a description will be given of the JPEG-DCT method as a first conventional example.
Before describing the first conventional example, a description will be given of DCT. The DCT which is used in image encoding is called two-dimensional DCT, to be accurate, and is obtained by independently processing two one-dimensional DCT in the horizontal direction and the vertical direction. According to xe2x80x9ckara seishi gazo no kokusai fugouka houshiki-JPEG arugorizumu- (International standard encoding method for color still image: JPEG Algorithm)xe2x80x9d (Endoh, Interface, 1991. 12, pp. 160-182), if an image block subject to transformation is written as x(m, n) and a transformed coefficient block as y(u, v), an 8xc3x978 DCT transformation formula and an inverse transformation formula for an 8-bit image can be written as follows.
[Mathematical Formula 1]                               y          ⁡                      (                          u              ,              v                        )                          =                                                            c                ⁡                                  (                  u                  )                                            ⁢                              c                ⁡                                  (                  v                  )                                                      4                    ⁢                                    ∑                              m                =                0                            7                        ⁢                          xe2x80x83                        ⁢                                          ∑                                  n                  =                  0                                7                            ⁢                              xe2x80x83                            ⁢                                                (                                                            x                      ⁡                                              (                                                  m                          ,                          n                                                )                                                              -                    128                                    )                                ⁢                cos                ⁢                                  xe2x80x83                                ⁢                                                                            (                                                                        2                          ⁢                          m                                                +                        1                                            )                                        ⁢                    u                    ⁢                                          xe2x80x83                                        ⁢                    π                                    16                                ⁢                                  xe2x80x83                                ⁢                cos                ⁢                                  xe2x80x83                                ⁢                                                                            (                                                                        2                          ⁢                          n                                                +                        1                                            )                                        ⁢                    v                    ⁢                                          xe2x80x83                                        ⁢                    π                                    16                                                                                        (        1        )                                                      x            ⁡                          (                              m                ,                n                            )                                =                                                    1                4                            ⁢                                                ∑                                      u                    =                    0                                    7                                ⁢                                  xe2x80x83                                ⁢                                                      ∑                                          v                      =                      0                                        7                                    ⁢                                      xe2x80x83                                    ⁢                                                            c                      ⁡                                              (                        u                        )                                                              ⁢                                          c                      ⁡                                              (                        v                        )                                                              ⁢                                          y                      ⁡                                              (                                                  u                          ,                          v                                                )                                                              ⁢                    cos                    ⁢                                          xe2x80x83                                        ⁢                                                                                            (                                                                                    2                              ⁢                              m                                                        +                            1                                                    )                                                ⁢                        u                        ⁢                                                  xe2x80x83                                                ⁢                        π                                            16                                        ⁢                                          xe2x80x83                                        ⁢                    cos                    ⁢                                          xe2x80x83                                        ⁢                                                                                            (                                                                                    2                              ⁢                              n                                                        +                            1                                                    )                                                ⁢                        v                        ⁢                                                  xe2x80x83                                                ⁢                        π                                            16                                                                                            +            128                          ⁢                  
                ⁢                                                                              where                  ⁢                                      xe2x80x83                                    ⁢                                      c                    ⁡                                          (                      u                      )                                                                      ,                                                      c                    ⁡                                          (                      v                      )                                                        =                                      xe2x80x83                                    ⁢                                                            1                                              2                                                              ⁢                                          xe2x80x83                                        ⁢                                          (                                              u                        ,                                                  v                          =                          0                                                                    )                                                                                                                                              =                                  xe2x80x83                                ⁢                                  1                  ⁢                                      xe2x80x83                                    ⁢                  and                  ⁢                                      xe2x80x83                                    ⁢                  others                                                                                        (        2        )            
FIGS. 29 and 30 are an image lossy encoding apparatus and an image lossy decoding apparatus, respectively, in accordance with the first conventional example. These drawings are partially taken from FIG. 3 on page 163 of xe2x80x9ckara seishi gazo no kokusal fugouka houshiki-JPEG arugorizumu- (International standard encoding method for color still image: JPEG Algorithm)xe2x80x9d (ibid.) and terms are modified. In FIGS. 29 and 30, reference numeral 10 denotes an image input unit; 20, a DCT unit; 35, a coefficient quantizing unit; 45, coefficient output unit; 110, input image data; 120, coefficient data; 170, quantized coefficient data; 225, a coefficient input unit; 240, an inverse DCT unit; 250, a decoded-image output unit; 260, a coefficient inversely-quantizing unit; 320, decoded image data; and 330, inversely-quantized coefficient data.
A description will be given of the various units shown in FIGS. 29 and 30. The encoding apparatus in FIG. 29 has the following configuration. The image input unit 10 receives as its input an image from an external circuit, and sends the same to the DCT unit 20 as the input image data 110. The DCT unit 20 effects DCT processing with respect to the input image data 110, and sends the result to the coefficient quantizing unit 30 as the coefficient data 120. The coefficient quantizing unit 30 effects quantization processing with respect to the coefficient data 120 in a predetermined method, and sends the result to a coefficient output unit 90 as the quantized coefficient data 170. The coefficient output unit 90 outputs the quantized coefficient data 170 to an external circuit.
Next, the decoding apparatus in FIG. 30 has the following configuration. The coefficient input unit 220 receives coefficients as its input, and sends the same to the coefficient inversely-quantizing unit 260 as the quantized coefficient data 170. With respect to the quantized coefficient data 170, the coefficient inversely-quantizing unit 260 effects inverse quantization, i.e., an inverse transformation of the quantization effected by the coefficient quantizing unit 30, and sends the result to the inverse DCT unit 240 as the inversely-quantized coefficient data 330. With respect to the inversely-quantized coefficient data 330, the inverse DCT unit 240 effects inverse DCT processing, i.e., an inverse transformation of the DCT processing effected-by the DCT unit 20, and sends the result to the decoded-image output unit 250 as the decoded image data 320. The decoded-image output unit 250 outputs the decoded image data 320 to-an external circuit.
The above-described configuration is a part of the first conventional example, and a general configuration is arranged such that the quantized coefficient data 170 is generally subjected to variable-length coding, such as Huffman coding and QM coding, by the encoding apparatus, while a decoding corresponding to the variable-length coding is effected by the decoding apparatus, thereby obtaining the quantized coefficient data 170. Since these portions are irrelevant to the essence of the present invention, and the omission of these portions does not impair the essence of the first conventional example, a description thereof will be omitted here.
On the basis of the above-described configuration, a description will be given of the operation of the first conventional example. FIGS. 31 and 32 are flowcharts illustrating the operation of the conventional example.
First, referring to FIG. 31, a description will be given of the encoding procedure in accordance with the first conventional example. In S10, an image is inputted to the image input unit 10 from an external circuit, and the input image data 110 is obtained. In S20, the DCT unit 20 effects DCT processing to obtain the coefficient data 120. In S35, the coefficient quantizing unit 30 effects quantization processing with respect to the coefficient data 120 in a predetermined method, thereby obtaining the quantized coefficient data 170. In S75, the coefficient output unit 90 outputs the quantized coefficient data 170 to an external circuit. In S80, a determination is made as to whether or not all the processing of the input image data 110 inputted has been completed, and if not completed, the operation returns to S10, and if completed, the encoding procedure ends.
Next, referring to FIG. 32, a description will be given of the decoding procedure in accordance with the first conventional example. In S115, coefficients are inputted to the coefficient input unit 220 from an external circuit, and the quantized coefficient data 170 is obtained. In S125, the coefficient inversely-quantizing unit 260 effects inverse quantization processing to obtain the inversely-quantized coefficient data 330. In S130, the inverse DCT unit 240 effects inverse DCT processing with respect to the inversely-quantized coefficient data, thereby obtaining the decoded image data 320. In S140, the decoded-image output unit 250 outputs the decoded image data 320 to an external circuit.
In S150, a determination is made as to whether or not all the processing of the quantized coefficient data 170 inputted has been completed, and if not completed, the operation returns to S115, and if completed, the decoding procedure ends.
A description will be given of the quantization processing which is effected by the coefficient quantizing unit 35 in the above-described operation. As described above, in a general frequency transform coding scheme, high-frequency components are coarsely quantized as compared with low-frequency components. In the JPEG-DCT method, a linear quantization of the formula shown below is used. Here, round is a function which returns an integer which is closest to an argument.
[Mathematical Formula 2]
(Quantization coefficient)=round ((DCT coefficient)/(quantization step)xe2x80x83xe2x80x83(3)
FIGS. 33A and 33B show a recommended quantization table of the JPEG-DCT method (source: ibid., FIG. 9 on page 167 of xe2x80x9ckara seishi gazo no kokusai fugouka houshiki-JPEG arugorizumu- (International standard encoding method for color still image: JPEG Algorithm)xe2x80x9d). In the drawing, numerals represent quantization steps, and the greater the numeral value, the more coarsely quantization is effected. In the same way as the DCT coefficients in Formula (1), the quantization table is written in such a manner that the frequency becomes higher from upper left toward lower right; hence, it follows that high-frequency components, in particular, are quantized coarsely.
Next, a conventional encoding technique concerning artificial images will be described as a second conventional example. In artificial images, since the same colors often locally appear in a space as shown in FIG. 28, predictive coding in which prediction of pixel values on the basis of neighboring pixels and encoding of prediction errors are combined is effective. Hereafter, as a typical example of predictive coding, the spatial method, which is a lossless coding method set forth in the aforementioned international JPEG, will be described as the second conventional example.
Before specifically describing the second conventional example, a description will be given of predictive coding. Predictive coding is a technique in which a pixel value of a pixel subjected to coding next is estimated, and a prediction error obtained by the following formula is encoded.
[Mathematical Formula 3]
xe2x80x83(Prediction error)=(actual pixel value)xe2x88x92(predicted value)xe2x80x83xe2x80x83(4)
In artificial images, since prediction errors are concentrated in 0 as shown in FIG. 27, the amount of codes can be reduced as compared with natural images. In addition, particularly in lossless predictive coding, amount-of-coding control cannot be effected; however, there is no possibility of the degradation of image quality.
Hereafter, a specific description will be given of the second conventional example. FIGS. 34 and 35 are an image lossy encoding apparatus and an image lossy decoding apparatus, respectively, in accordance with the second conventional example. These drawings are partially taken from ibid., FIG. 17 on page 173 of xe2x80x9ckara seishi gazo no Kokusai fugouka houshiki-JPEG arugorizumu- (International standard encoding method for color still image: JPEG Algorithm)xe2x80x9d, a decoding apparatus is added, and terms are modified. In the drawings, those portions which are similar to those of FIGS. 29 and 30 will be denoted by the same reference numerals, and a description thereof will be omitted. Reference numeral 25 denotes a predicting unit; 46, a prediction-error output unit; 226, a prediction-error input unit; and 171, prediction error data.
A description will be given of the various units shown in FIGS. 34 and 35. The encoding apparatus in FIG. 34 has the following configuration. The predicting unit 25 predicts a pixel value to be encoded next by using the input image data 110, and sends a difference with an actual pixel value to the prediction-error output unit 46 as the prediction error data 171.
The decoding apparatus in FIG. 35 has the following configuration. The prediction-error input unit 226 receives prediction errors as its input, and sends the same to the predicting unit 25 as the prediction error data 171. Although the predicting unit 25 is identical to the predicting unit 25 of the encoding apparatus, but differs from the same in that reference is made to a decoded image for predicting a next pixel.
On the basis of the above-described configuration, a description will be given of the operation in accordance with the second conventional example. FIGS. 36 and 37 are flowcharts illustrating the operation of the conventional example.
First, referring to FIG. 36, a description will be given of the encoding procedure in accordance with the first conventional example. Those portions which are similar to those of FIG. 31 are denoted by the same reference numerals, and a description thereof will be omitted. In S25, the predicting unit 25 computes a prediction error in accordance with Formula (4). In S76, the prediction-error output unit 46 outputs to an external circuit the prediction error data 171 computed in S25.
Next, referring to FIG. 37, a description will be given of the decoding procedure in accordance with the first conventional example. Those portions which are similar to those of FIG. 32 are denoted by the same reference numerals, and a description thereof will be omitted. In S116, the prediction-error input unit 226 receives as its input the prediction error from an external circuit. In S135, the predicting unit 25 computes a pixel value by the addition of the predicted value and the prediction error.
A description will be given of the prediction error computation processing in the description of the operation. In the JPEG-Spatial method, it is decided that one of the seven predictors shown in FIG. 38 be used. For example, in a case where a is selected as a prediction formula, it suffices if the value of a pixel which is a left-side neighbor to a pixel x to be encoded from now on is set as a predicted value.
Although the first and second conventional examples have been described above, it is shown below that it is difficult to effect encoding efficiently by either one of the first and second conventional examples irrespective of the distinction between natural images and artificial images.
Since, in an artificial image, important information is included in high-frequency components as well, if quantization is effected as shown in FIGS. 33A and 33B in which a high-frequency region is coarsely quantized, the degradation of the image quality, e.g., mosquito noise, occurs. An example of mosquito noise which occurred due to the quantization table shown in FIG. 33A is shown in FIGS. 39A and 39B. FIG. 39A shows an input image, while FIG. 39D shows a decoded image. In a frequency transform coding scheme such as JPEG-DCT, such noise makes it difficult to reduce the amount of codes while maintaining the image quality with respect to an artificial image. This state is shown in FIG. 40.
On the other hand, in the case of a natural image, pixel values differ even among neighboring pixels due to the effect of noise, so that the amount of codes does not diminish in the lossless predictive coding such as JPEG-Spatial method. This state is shown in FIG. 41. In addition, in the lossless coding, the image quality and the amount of codes cannot be traded off, so that amount-of-coding control cannot be effected. Since this drawback directly affects the capacity of a storage medium, a communication band, and the like, the structuring of the system is made difficult.
Thus, in the first and second conventional examples, images which cannot be encoded effectively are present. To overcome this problem, a technique in which lossy encoding and lossless encoding are selectively used for respective portions is conceivable. As such an example, Japanese Patent Application Laid-Open No. 113145/1994 is known. Hereafter, the invention disclosed in that publication will be described as a third conventional example.
FIG. 42 is a schematic diagram of an image processing apparatus in accordance with the third conventional example. In this drawing, a portion of FIG. 1 in that publication is omitted in such a way as not to impair the purport of Japanese Patent Application Laid-Open No. 113145/1994, and terms are modified. In the drawing, reference numeral 15 denotes an artificial-image input unit; 16, a natural-image input unit; 90, an artificial-image encoding unit; 91, a natural-image encoding unit; 92, an artificial-image storage unit; 93, a natural-image storage unit; 94, an artificial-image decoding unit; 95, a natural-image decoding unit; 96, an image composing unit; 112, input artificial-image data; 113, input natural-image data; 114, encoded artificial-image data; 115, encoded natural-image data; 116, decoded artificial-image data; and 117, decoded natural-image data.
A description will be given of the various units shown in FIG. 42. the artificial-image input unit 15 and the natural-image input unit 16 respectively receive as their inputs an artificial image and a natural image from external circuits, and send them to the artificial-image encoding unit 90 and the natural-image encoding unit 91 as the input artificial-image data and the input natural-image data 113, respectively. The artificial-image encoding unit 90 and the natural-image encoding unit 91 effect encoding with respect to the input artificial-image data and the input natural-image data 113 by predetermined techniques, respectively, and send the results to the artificial-image storage unit 92 and the natural-image storage unit 93 as the encoded artificial-image data 114 and the encoded natural-image data 115, respectively. The artificial-image storage unit 92 and the natural-image storage unit 93 temporarily store the encoded artificial-image data 114 and the encoded natural-image data 115, respectively, and send them to the artificial-image decoding unit 94 and the natural-image decoding unit 95, respectively. With respect to the encoded artificial-image data 114 and the encoded natural-image data 115, the artificial-image decoding unit 94 and the natural-image decoding unit 95 respectively effect decoding processings corresponding to the encoding effected by the artificial-image encoding unit 90 and the natural-image encoding unit 91, and send the results to the image composing unit 96 as the decoded artificial-image data 116 and the decoded natural-image data 117, respectively. The image composing unit 96 combines the decoded artificial-image data 116 and the decoded natural-image data 117.
In the above description, it is stated in the first embodiment of the aforementioned patent that the encoding which is effected by the artificial-image encoding unit 90 xe2x80x9chas the function of a lossless method such as the run-length coding method.xe2x80x9d In addition, it is also stated in the first embodiment of that patent that the encoding which is effected by the natural-image encoding unit 91 is xe2x80x9can image compression system such as JPEGxe2x80x9d It should be noted that JPEG referred to in that patent refers to the JPEG-DCT method referred to in this description.
It has already been pointed out that since the first and second conventional examples are designed specifically for natural images and artificial images, respectively, it is difficult to handle both types of images efficiently by either one of the independent techniques.
In the third conventional example, since a natural image and an artificial image are encoded and decoded in parallel by totally different methods, the processing times in both processings generally do not coincide. For this reason, it is impossible to produce an output to an external circuit until all the encoded data are gathered during encoding, and until all the image data are gathered during decoding. Hence, at least one image portion of a code buffer is required for the encoding apparatus, while at least one image portion of an image buffer is similarly required for the decoding apparatus. These are unnecessary configurations in the case of an image encoding/decoding apparatus having a method of only one system.
In addition, since both the encoding apparatus and the decoding apparatus have two systems or more, an increase in the scale of the apparatus results. Further, since the image is expressed by a plurality of totally different codes, the handling of codes becomes complex during such as transmission or storage. Still further, with respect to the image quality of a decoded image as well, noise can possibly occur in a portion where the encoding method is changed over.
The present invention has-been devised in view of the above-described circumstances, and its object is to provide a single encoding apparatus and a decoding apparatus capable of effective compression irrespective of the distinction between a natural image and an artificial image.
To attain the above object, the present invention adopts the following configurations. First, a description will be given of the invention of the image encoding apparatus.
In accordance with the invention, there is provided an image encoding apparatus comprising: image input means for inputting an image; frequency transforming means for effecting frequency transform for determining frequency components of the image inputted by said image input means; threshold processing means for effecting threshold processing of the frequency components determined by said frequency transforming means; low-frequency image output means for outputting an image of low-frequency components of the image inputted by said image input means, in correspondence with a result of threshold processing by said threshold processing means; pixel subsampling means for effecting predetermined subsampling processing with respect to the image outputted by said low-frequency image output means, in correspondence with the result of threshold processing by said threshold processing means; coefficient-information output means for outputting the result of threshold processing by said threshold processing means; and subsampled-image output means for outputting the image subjected to subsampling processing by said pixel subsampling means.
In this configuration, by representing an image with an optimum resolution, it is possible to suppress redundant components and reduce the amount of codes. To determine the optimum resolution, frequency analysis is performed, and the subsampling processing of pixels is effected on the basis of the result of this analysis.
In accordance with the invention, there is provided an image encoding apparatus comprising: image input means for inputting an image; frequency transforming means for effecting frequency transform for determining frequency components of the image inputted by said image input means; threshold processing means for effecting threshold processing of the frequency components determined by said frequency transforming means; high-frequency coefficient masking means for replacing high-frequency components with 0s of the frequency components determined by said frequency transforming means, in correspondence with a result of threshold processing by said threshold processing means; inversely transforming means for effecting inverse frequency transform in which the frequency components with the high-frequency components replaced into 0s by said high-frequency coefficient masking means are converted into an image; pixel subsampling means for effecting predetermined subsampling processing with respect to the image converted by said inversely transforming means, in correspondence with the result of threshold processing by said threshold processing means; coefficient-information output means for outputting the result of threshold processing by said threshold processing means; and subsampled-image output means for outputting the image subjected to subsampling processing by said pixel subsampling means.
In this configuration as well, by representing an image with an optimum resolution, it is possible to suppress redundant components and reduce the amount of codes.
In accordance with the invention, there is provided an image encoding apparatus comprising: image input means for inputting an image; frequency transforming means for effecting frequency transform for determining frequency components of the image inputted by said image input means; threshold processing means for effecting threshold processing of the frequency components determined by said frequency transforming means; pixel subsampling means for effecting predetermined subsampling processing with respect to the image inputted by said image input means, in correspondence with the result of threshold processing by said threshold processing means; coefficient-information output means for outputting the result of threshold processing by said threshold processing means; and subsampled-image output means for outputting the image subjected to subsampling processing by said pixel subsampling means.
In this configuration as well, by representing an image with an optimum resolution, it is possible to suppress redundant components and reduce the amount of codes.
In accordance with the invention, there is provided an image encoding apparatus comprising: image input means for inputting an image; pseudo-decoded-image generating means for generating a pseudo-decoded image by subjecting the image inputted by said image input means to predetermined subsampling processing and predetermined interpolation processing; coefficient analyzing means for determining a subsampling rate on the basis of an error between the pseudo-decoded image generated by said pseudo-decoded-image generating means and the image inputted by said image input means; pixel subsampling means for effecting predetermined subsampling processing with respect to the image inputted by said image input means, in correspondence with the subsampling rate determined by said coefficient analyzing means; coefficient-information output means for outputting the subsampling rate determined by said coefficient analyzing means; and subsampled-image output means for outputting the image subjected to subsampling processing by said pixel subsampling means.
In this configuration as well, by representing an image with an optimum resolution, it is possible to suppress redundant components and reduce the amount of codes.
Further, in accordance with the invention, in the image encoding apparatus, the error used in said coefficient analyzing means is a maximum value of a pixel value error, an absolute value of the error, and a squared value of the error, or one of a dynamic range, a variance, and an SN ratio.
Further, in accordance with the invention, in the image encoding apparatus, the predetermined interpolation processing by said pseudo-decoded-image generating means is one of nearest-neighbor interpolation, 4-point linear interpolation, 9-point second-order interpolation, cubic convolution interpolation, and low-pass filter processing.
In accordance with the invention, there is provided an image encoding apparatus comprising: code input means for inputting codes obtained by subjecting an image to frequency transform and entropy coding; entropy decoding means for obtaining frequency components by subjecting the codes inputted by said code input means to decoding which is an inverse transformation of entropy coding effected with respect to the codes; threshold processing means for effecting threshold processing with respect to the frequency components obtained by said entropy decoding means; high-frequency coefficient masking means for replacing high-frequency components with 0s of the frequency components obtained by said entropy decoding means, in correspondence with a result of threshold processing by said threshold processing means; inversely transforming means for effecting inverse frequency transform in which the frequency components with the high-frequency components replaced into 0s by said high-frequency coefficient masking means are converted into an image; pixel subsampling means for effecting predetermined subsampling processing with respect to the image converted by said inversely transforming means, in correspondence with the result of threshold processing by said threshold processing means; coefficient-information output means for outputting the result of threshold processing by said threshold processing means; and subsampled-image output means for outputting the image subjected to subsampling processing by said pixel subsampling means.
Further, in accordance with the invention, in the image encoding apparatus, the decoding by said entropy decoding means is one of Huffman coding, arithmetic coding, and QM coding.
In accordance with the invention, there is provided an image encoding apparatus comprising: image input means for inputting an image; frequency transforming means for effecting frequency transform for determining frequency components of the image inputted by said image input means; threshold processing means for effecting threshold processing of the frequency components determined by said frequency transforming means; high-frequency coefficient masking means for replacing high-frequency components with 0s of the frequency components obtained by said frequency transforming means, in correspondence with a result of threshold processing by said threshold processing means; inversely transforming means for effecting inverse frequency transform in which the frequency components with the high-frequency components replaced into 0s by said high-frequency coefficient masking means are converted into an image; pixel subsampling means for effecting predetermined subsampling processing with respect to the image converted by said inversely transforming means, in correspondence with the result of threshold processing by said threshold processing means; data composing means for combining the subsampled image obtained by said pixel subsampling means and the result of threshold processing by said threshold processing means; and composite-data output means for outputting composite data composed by said data composing means.
In this configuration as-well, by representing an image with an optimum resolution, it is possible to suppress redundant components and reduce the amount of codes.
In accordance with the invention, there is provided an image encoding apparatus comprising: image input means for inputting an image; coefficient-information input means for inputting coefficient information; frequency transforming means for effecting frequency transform for determining frequency components of the image inputted by said image input means; high-frequency coefficient masking means for replacing high-frequency components with 0s of the frequency components determined by said-frequency transforming means, in correspondence with the coefficient information inputted by said coefficient-information input means; inversely transforming means for effecting inverse frequency transform in which the frequency components with the high-frequency components replaced into 0s by said high-frequency coefficient masking means are converted into an image; pixel subsampling means for effecting predetermined subsampling processing with respect to the image converted by said inversely transforming means, in correspondence with the coefficient information inputted by said coefficient-information input means; coefficient-information output means for outputting the coefficient information inputted by said coefficient-information input means; and subsampled-image output means for outputting the image subjected to subsampling processing by said pixel subsampling means.
In this configuration as well, by representing an image with an optimum resolution, it is possible to suppress redundant components and reduce the amount of codes.
Further, in accordance with the invention, the image encoding apparatus further comprises: image encoding means for effecting image encoding with respect to the subsampled image outputted by said subsampled-image output means.
Further, in accordance with the invention, in the image encoding apparatus, the image encoding effected by said image encoding means is one of or both of lossless coding and predicting coding.
Further, in accordance with the invention, the image encoding apparatus further comprises: coefficient-image encoding means for effecting entropy coding with respect to the coefficient image outputted by said coefficient-image output means.
Further, in accordance with the invention, in the image encoding apparatus, the frequency transform effected by said frequency transforming means and said inversely transforming means is one of discrete cosine transform, Fourier transform, discrete sine transform, subband transform, and wavelet transform.
Further, in accordance with the invention, in the image encoding apparatus, the threshold processing by said threshold processing means is threshold processing in which a predetermined quantization table is set as the threshold.
Further, in accordance with the invention, in the image encoding apparatus, the quantization table used by said threshold processing means can be set by an external circuit.
Further, in accordance with the invention, in the image encoding apparatus, said high-frequency coefficient masking means replaces a component greater than a maximum frequency component with a 0 by means of said threshold processing means.
Further, in accordance with the invention, in the image encoding apparatus, the subsampling processing by said pixel subsampling means is effected in proportion to a ratio which is derived from a distribution of maximum frequencies within a block or frequency components which are not 0s.
Further, in accordance with the invention, in the image encoding apparatus, a ratio of subsampling processing effected by said pixel subsampling means is quantization to a predetermined value set in advance.
Further, in accordance with the invention, in the image encoding apparatus, the predetermined subsampling processing by said pixel subsampling means is one of leaving pixels in lattice form, effecting the subsampling processing at identical rates for a vertical direction and a horizontal direction, effecting the subsampling processing such that pixels which remain become substantially equidistanced, and preferentially leaving peak values in neighboring pixels.
Further, in accordance with the invention, in the image encoding apparatus, the subsampling processing by said pixel subsampling means is the thinning out of the same pixels which were previously thinned out in a case where the image inputted by said image input means were already subjected to encoding by said image encoding apparatus.
Further, in accordance with the invention, the image encoding apparatus further comprises: pixel-value quantizing means for quantizing a pixel value of the image subjected to subsampling processing by said pixel subsampling means.
Further, in accordance with the invention, in the image encoding apparatus, said pixel-value quantizing means changes a quantization step in correspondence with a result of threshold processing by said threshold processing means, or changes the quantization step in correspondence with a magnitude of the threshold used by said threshold processing means.
Further, in accordance with the invention, the image encoding apparatus further comprises: image determining means for determining the threshold used by said threshold processing means by performing predetermined analysis with respect to the image inputted by said image input means.
Further, in accordance with the invention, in the image encoding apparatus, said image determining means determines a difference between a natural image and an artificial image, and in the case of the artificial image sets the threshold to a 0 and effects control so as to prevent the occurrence of a frequency component which is set to a 0 in the threshold processing by said threshold processing means.
Further, in accordance with the invention, in the image encoding apparatus, the predetermined analysis processing by said image determining means involves measurement of a dynamic range of the pixel values, measurement of a histogram of the pixel values, measurement of an entropy of lower bits of the pixel values, measurement of the sharpness of an edge, measurement of the size of a line, measurement of a frequency component, designation from an external circuit, and detection of at least one component from among an edge, a pattern, a gradation, and a line.
Next, a description will be given of an image decoding apparatus.
In accordance with the invention, there is provided an image decoding apparatus comprising: coefficient-information input means for inputting coefficient information; subsampled-image input means for inputting a subsampled image; coefficient interpolating means for computing a frequency component by a predetermined technique in correspondence with the subsampled image inputted by said subsampled-image input means and the coefficient information inputted by said coefficient-information input means; inversely transforming means for effecting inverse frequency transform so as to convert the frequency component computed by said coefficient interpolating means into an image; and decoded-image output means for outputting the image converted by said inversely transforming means.
In this configuration, it is possible to decode image data which has been compressed by effecting adaptive subsampling in correspondence with frequency analysis.
In accordance with the invention, there is provided an image decoding apparatus comprising: coefficient-information input means for inputting coefficient information for each block which is a fixed region of an image; subsampled-image input means for inputting a subsampled image for each block; pixel-value interpolating means for interpolating a pixel value by a predetermined technique in correspondence with the subsampled image inputted by said subsampled-image input means and the coefficient information inputted by said coefficient-information input means; and decoded-image output means for outputting the image interpolated by said pixel-value interpolating means.
In this configuration as well, it is possible to decode image data which has been compressed by effecting adaptive subsampling in correspondence with frequency analysis.
Further, in accordance with the invention, in the image decoding apparatus, the predetermined technique used by said pixel-value interpolating means is one of nearest-neighbor interpolation, 4-point linear interpolation, 9-point second-order interpolation, cubic convolution interpolation, and low-pass filter processing.
In accordance with the invention, there is provided an image decoding apparatus comprising: composite-data input means for inputting composite data which is data combining coefficient information and a subsampled image; data decomposing means for decoding the composite data inputted by said composite-data input means into the subsampled image and the coefficient information; coefficient interpolating means for computing a frequency component by a predetermined technique, in correspondence with the subsampled image and the coefficient information which were decomposed by said data decomposing means; inversely transforming means for effecting inverse frequency transform in which the frequency component computed by said coefficient interpolating means is converted into the image; and decoded-image output means for outputting the image converted by said inversely transforming means.
In this configuration as well, it is possible to decode image data which has been compressed by effecting adaptive subsampling in correspondence with frequency analysis.
Further, in accordance with the invention, the image decoding apparatus further comprises: image decoding means for decoding into an image a code subjected to image encoding with respect to the subsampled image, wherein said subsampled-image input means inputs as the subsampled image the image decoded by said image decoding means.
Further, in accordance with the invention, in the image decoding apparatus, the decoding effected by said image decoding means is inverse processing of lossless coding or inverse processing of predictive coding.
Further, in accordance with the invention, the image decoding apparatus further comprises: pixel-value correcting means for replacing a pixel, which is included in the subsampled image inputted by said subsampled-image input means of the image converted by said inversely transforming means, with the pixel value of the subsampled image, wherein said decoded-image output means outputs the image corrected by said pixel-value correcting means.
Further, in accordance with the invention, in the image decoding apparatus, the frequency transform effected by said inversely transforming means and said inversely transforming means is one of discrete cosine transform, Fourier transform, discrete sine transform, subband transform, and wavelet transform.
Further, in accordance with the invention, in the image decoding apparatus, the coefficient interpolation effected by said coefficient interpolating means is one of the solving of a simultaneous system of linear equations concerning frequency coefficients and pixel values, computation of an inverse matrix determined in advance with respect to the simultaneous system of linear equations concerning frequency coefficients and pixel values, and low-pass filtering of the subsampled image or approximate processing.
In accordance with the invention, there is provided an image encoding/decoding apparatus comprising: image input means for inputting an image; frequency transforming means for effecting frequency transform for determining frequency components of the image inputted by said image input means; threshold processing means for effecting threshold processing of the frequency components determined by said frequency transforming means; high-frequency coefficient masking means for replacing high-frequency components with 0s of the frequency components determined by said frequency transforming means, in correspondence with a result of threshold processing by said threshold processing means; first inversely transforming means for effecting inverse frequency transform in which the frequency components with the high-frequency components replaced into 0s by said high-frequency coefficient masking means are converted into an image; pixel subsampling means for effecting predetermined subsampling processing with respect to the image converted by said first inversely transforming means, in correspondence with the result of threshold processing by said threshold processing means; coefficient-information output means for outputting the result of threshold processing by said threshold processing means; subsampled-image output means for outputting the image subjected to subsampling processing by said pixel subsampling means; coefficient-information input means for inputting coefficient information which is a result of threshold processing outputted by said coefficient-information output means; subsampled-image input means for inputting the subsampled image outputted by said subsampled-image output means; coefficient interpolating means for computing a frequency component by a predetermined technique in correspondence with the subsampled image inputted by said subsampled-image input means and the coefficient information inputted by said coefficient-information input means; second inversely transforming means for effecting inverse frequency transform so as to convert the frequency component computed by said coefficient interpolating means into an image; and decoded-image output means for outputting the image converted by said second inversely transforming means.
In this configuration, by representing an image with an optimum resolution, it is possible to suppress redundant components and reduce the amount of codes. To obtain the optimum resolution, frequency analysis is performed, and the subsampling processing of pixels is effected on the basis of the result of this analysis. In addition, it is possible to decode image data which has been compressed by effecting adaptive subsampling in correspondence with the frequency analysis.
In accordance with the invention, there is provided an image encoding method comprising: step 1 for inputting an image; step 2 for effecting frequency transform for determining frequency components of the image inputted in said step 1; step 3 for effecting threshold processing of the frequency components determined in said step 2; step 4 for replacing high-frequency components with 0s of the frequency components determined in said step 2, in correspondence with a result of threshold processing in said step 3; step 5 for effecting inverse frequency transform in which the frequency components with the high-frequency components replaced into 0s in said step 4 are converted into an image; step 6 for effecting predetermined subsampling processing with respect to the image converted in said step 5, in correspondence with the result of threshold processing in said step 3; step 7 for outputting the result of threshold processing in said step 3; and step 8 for outputting the image subjected to subsampling processing in said step 6.
In this configuration, by representing an image with an optimum resolution, it is possible to suppress redundant components and reduce the amount of codes.
In accordance with the invention, there is provided an image decoding method comprising: step 1 for inputting coefficient information; step 2 for inputting a subsampled image; step 3 for computing a frequency component by a predetermined technique in correspondence with the subsampled image inputted in said step 2 and the coefficient information inputted in said step 1; step 4 for effecting inverse frequency transform so as to convert the frequency component computed said step 3; and step 5 for outputting the image converted step 4.
In this configuration as well, it is possible to decode image data which has been compressed by effecting adaptive subsampling in correspondence with the frequency analysis.